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Amendments to the Specification 

The following new paragraph is ADDED after the paragraph that begins at page 35, line 12 
with "The OEVI further similarly implements": 

"As shown in the FIG. 12c example, a user pointing in the email pane of 
window 1203 (i.e., pointing at a particular email) may utilize such commands to cause 
scrolling of various window portions, e.g., horizontal or vertical scrolling of the email 
window portion 1 23 1 a, the preview pane 1 23 1 b or the folder pane 1231c. In this example, 
each command further provides for returning to the pointing at the particular email (e.g., 
1231c, 1232b, 1232c)." 

The following 6 new consecutive paragraphs are ADDED after the paragraph that begins at 
page 74, line 3 with "A process by which voice-command elements can be derived. . 

"FIGS. 9a through 10b illustrate further interfacing. system embodiments 
according to the invention. FIG. 9a, for example, illustrates an interfacing system 900 in 
which user 901 is using single-user machine 910 which includes an interface shown as 
voice interface 911. Similarly, in FIG. 9b, a user is using one or more of local system 921, 
remote system 922, and remote system 923. As shown, the user uses local machine 921 
directly, remote machines 922 and 923 via the local interface of local system 921 and the 
remote interfaces of remote machines 922 and 923, and remote machine 923 via the 
coupling between local system 921 and remote system 923. In interfacing system 940 of 
FIG. 9c, the user uses one or more of underlying application(s) 933. As shown, user 
commands/dictation are received by voice interface 931 which is coupled via one or more 
. speech program(s) 932 to one or more underlying application(s) 933. Voice interface 931 
is further coupled to non-voice interface 934 and speech program(s) 932 is ftirther coupled 
to I/O interface 935. Underlying application(s) 933 may also receive user input via non- 
voice interface 934 and I/O interface 935 (or via the above further couplings), and may 
provide output to the user via I/O interface 935 and non- voice interface 934 or via the 
above further couplings. 
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Interfacing system 940 of FIG, 9d also illustrates the user using one or more 
underlying application(s). This example, however, illustrates that user input is digitized by 
digitizer 941 and may be recorded by or bypass recorder 942. No directional arrows are 
indicated and underlying application(s) 944 may receive recognized input from one or 
more of recognition engines 943 or command-interpreted input from command interpreter 
947. (Interpreter may also operate in conjunction with voice interface 948, which includes 
voice command information 948a and operational code information 948b.) Underlying 
applications 944 are also coupled via command interpreter 947 (which is further coupled to 
the user via recognition engines 943 and which engines 943 are also coupled to recorder 
942), synthesis engine 946 (which is also coupled to recorder 942) and D/A converter 945. 
FIG. 9d does not, however, exclude other examples that are also depicted by FIG. 9d. (For 
example, recorder 942 is also coupled to underlying applications 944 via recognition 
engines 943 and command interpreter 947.) System 950 of FIG. 9e illustrates how 
information may be exchanged by a local system 951 /speech engine of the local system 
to/from components of remote systems 952-956 that are coupled to the local system 951 
(see directional arrows). The remote system component examples illustrated by system 
950 include an application of remote system 952, a recorder of remote system 953, a 
transcriber of remote system 954, a translater of remote system 955 and parameters, e.g., of 
a user, stored by remote system 956. 

The interfacing system 1000 example of FIG. 10a includes similar 
components as with system 940 of FIG. 9d. However, while system 1000 receives from a 
user conversant commands and dictation (e.g., conversant), a command interpreter is not 
explicitly depicted. In this example, an underlying application, e.g., of a host, includes 
database 1003a and interface 1003b, an example of which is depicted as interface 1003c. 
Interface 1003c includes various screens, each screen including interface regions and 
objects (e.g., lOA-lOD and lOE respectively). As shown, user conversant 
commands/dictation may be digitized by digitizer 1001, recorded by recorder 1004 or both. 
Digitizer 1001 /recorder 1004 output is received by recognition engine 1002 and recognized 
such information is provided to underlying (host) app 1003 or interface 1003b. Interface 
1003b may also receive input from other I/O services via non-speech interface 1007. FIG. 
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10a also shows that underlying (host) app 1003 or interface 1003b output may be provided 
to speech synthesis engine 1006 or further components coupled thereto, and may be 
provided to other I/O services via non-speech interface 1007. 

FIG. 10b illustrates an interfacing system 1010 example in which multiple 
users are depicted as using various devices for conducting workgroup or other interactions. 
As shown, for example, users 101 la2 and 101 la3 are both using local system- 1, which 
includes conversant interface 1012, and users 101 la2 and 101 la3 are also using portable 
computing device 1011b and mouse 1011c respectively. User 101 lal is using Local 
system-N 1013, user 101 la4 is using smart phone 1018d, users 101 la5-6 are using video 
conferencing machine 1018 and user 101 la7 is using tablet 101 la7. Various user devices 
are also remotely coupled to video conferencing machine 1018, as well as to shared I/O 
1014, stereo, phone or production system 1017, translator 101 8g and machine-N, which 
provides recorder/transcriber 101 8e. Also coupled via a network are user-2 machine 1015 
storing speech information, server 1016 storing user information and user- 1 machines 
1 0 1 8f (with information to present. Cameras 1 0 1 8e are further shown as proximate to 
movement of user 101 la6 (as shown by the illustrated directional arrow); in contrast to 
users 101 la6-7, however, user 101 la5 is shown as not moving and a corresponding camera 
captures the user's face, which is further shown in interface lOlScl. 

An interface conversion system example is further illustrated by FIG. 10c. 
In the interface conversion system 1020 of FIG. 10c, interface conversion is performed by 
an interface conversion engine, and more specifically, voice interface conversion engine 
1023. As shovm from left to right, conversational commands are received by system 1020 
that include tasks al-N 1021a, 1021b having a context A 1022a and tasks bl-N 1021c, 
102 Id having a context B 1022b. Also received by voice interface conversion engine 1023 
are inter-context tasks 1021e5 which voice interface conversion engine 1023 converts and 
outputs to a coupled voice command interpreter. FIG. 1 OC further shows how, looking at 
system 1020 instead from right to left, the process would be reversed. 

FIG. lOd further illustrates an exemplary new command converter. 
Converter control 1031 of new command converter 1030 controls host analyzer 1032 and 
host operation analyzer/mapper 1032. Host analyzer 1032a analyzes a host interface, e.g., 



- 5 of 69 - 



Appl. No. 09/905,590 

RCE and Response to Office Action mailed April 1, 2009 



PATENT 



using an interface mapper to map a host interface. Approach analyzer 1033a further 
analyzes an applicable approach and conversant factor checker checks application of 
conversant factors, thereby producing an interface output. Additionally, host operation 
analyzer map 1032b correspondingly analyzes/maps applicable host operation, underlying 
app check/map 1033d checks/maps an applicable underlying application and optimizer 
1035 provides for optimizing the result (e.g., compiling), thereby producing an applicable 
control output. Each of the aforementioned system 1030 components is further coupled to 
and is operable in conjunction with iteration check 1033b and knowledge base 1033c." 

The following new paragraph is ADDED after the paragraph that begins at page 100, line 
13 with "Verbiage is further preferably the same or similar. . .": 

"FIGS. 12el and 12e2 further illustrate simple and complex paging 
examples. These examples depict only 2 elements, i.e., containers and information 
contained inside the containers. In FIG. 12el, for example, arrows la and lb respectively 
show how simple paging may be used for redirecting from referring to a first container 
(e.g., file folder 1205) or information within a first file folder 1251 to referring to a second 
container (e.g., file folder 1252). FIG. 12el also shows how complex paging further 
provides for redirecting from the first container (file folder 1205) or information within a 
first file folder 1251 to referring to information 1253 inside file folder 1252. The FIG. 
1 2e2 example instead illustrates a windowing interface, and should be recognized as a 
portion of the main OE window. (Paging may, for example, be conducted using the 
command "GOTO <foldemame> (messages) and is illustrated using folders, but paging 
may also apply to any such re-direction.) Thus, this example provides for switching 
referencing from an OE folder object 1205a (e.g. inbox) or one of a list of email identifiers 
1251a (e.g., specified as outbox messages) to a different folder object 1252 (e.g., specified 
as business folder) or email identifiers inside folder object 1252 (e.g., specified as business 
messages)." 

The following new paragraph is ADDED after the paragraph that begins at page 95, line 20 
with "Efficient communicative voice-commands employing inter-item designation 
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"FIG. 12f illustrates an example of how selection and repositioning may be 
conducted using the terms "last", "these last", "these (next)" and "next". In the example 
shown, a current designation is given by "a" references (2106a, 1262a, 1263a, 1264a), a 
selection is given by "b" references (1206b, 1262b, 1263b, 1264b) and a successive 
designation is given by "c" references (1206c, 1262c, 1263c, 1264c). In the FIG. 12h 
example, the command "Move next three to Business folder" is used, from a current email 
indicator ("current msg") in an email pane, to select the corresponding emails ("next", 
"three"), which are darkened. The command is provides for a "move" of the selected 
emails to the corresponding destination ("Business folder" in the folder pane) and feedback 
(here, returning to the email pane and to a corresponding email indicator in the email 
pane." 

The following new paragraph is ADDED after the paragraph that begins at page 108, line 3 
with "Following selection, "And move" and/or "And select" commands . . .": 

"FIG. 12g illustrates an example in which designation (here, selection, 
repositioning and linking) is conducted in a consistent manner in conjunction with different 
underlying interface elements (e.g., windows) 1270 and 1270a, arid despite the depicted 
different underlying interface types and uses that should be familiar to OE users. In the 
window 1270 example, a current email indicator is selected (e.g., "Select..."), a link plus 
operability (e.g., "And move. . .") retains a previous selection and moves to a corresponding 
email indicator, and a further link plus operability (e.g., "And select. . .") again retains the 
previous selection and conducts a further selection indicated by the darkened email 
indicators and arrow. In the window 1270a example, "Select. . ." selects the corresponding 
object (e.g., a presented Name input field), "name" enters "Name" in the input field, a link 
plus operability (e.g., "And phone <No.>") retains a previous selection, moves to a 
corresponding object (e.g. phone number input field) arid enters the <No,> into the field. 
In each of the examples, a further operability affects the selection, e.g., flagging the 
selected email indicators or finding data corresponding to the entered name plus phone 
respectively." 
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The following new paragraph is ADDED after the paragraph that begins at page 97, line 8 
with "Reference-feature and reference-item methods can also be combined 

"In the FIG. 13a examples, current and non-current underlying interface 
portions 1301b, 1301c (e.g., OE find people and select recipients windows respectively) are 
operated together in a consistent manner as with the singular underlying interface portion 
1301d (e.g., OE new message window). As shown by arrows 1311 in the 1301a example, 
starting with a current designation in window 1301b, a user may utilize a corresponding 
command to add selected addressees from the selection field of window 1301b to selected 
ones of the output selection fields 1314b of window 1301c (e.g., addressee fields "to", "cc" 
or "bcc"), in conjunction with which the user is retumed to the current designation (arrow 
1312). A user may also utilize a corresponding command to modify previous selection, for 
example, deleting selected addressees fi"om the output selection fields 1314b of window 
1301c (see arrows 131 1), in conjunction with which the user is also retumed to the current 
designation (arrow 1312). Example 1301a also illustrates the use of nth terminology 1314a 
(e.g., 1st, 2nd 3rd. . .) in conjunction with such commands and how indicators of such or 
separation indicators may also be added to and presented as part of the underlying 
interface. Examples of anti-aliases are also shown. A user may also similarly utilize a 
command for adding to or modifying selected addressees or attachments in the window 
1 30 Id example. A user may, for example, starting with a current designation of a portion 
of window 130 Id (e.g., message body), add or delete selected addressee or attachment 
information using a similar command as with windows 1301b-c (arrows 1313), in 
conjunction with which the user is retumed to the current designation. The use of nth and 
other terminology (e.g., and, thru, all) or addition of the same to window 130 Id is also 
depicted." 

The following new paragraph is ADDED after the paragraph that begins at page 105, line 5 
with "The above window referencing methods can also be used . . 

"FIG. 14 illustrates an example of "merge and reply" or "merge and 
forward" commands. In this example, contact information may be retrieved from 
local/remote machines (e.g., local/remote address books coupled to OE) or from subject 
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messages, e.g., copying corresponding data from email windows 1401b, 1401c (line 1415a 
and arrows 1515b, 1412). Message information may be retrieved via corresponding email 
indicators, e.g., by opening corresponding messages indicated by window 1401a (arrow 
1411) and copying the email information from the resulting open windows 1401b-c to the 
message body portion of window 1401d (arrows 1413a-b). Cycling/collecting data may be 
used as needed (arrow 1414), and annotation or added indicators (e.g., separator 1413c) 
may also be used." 

The following new paragraph is ADDED after the paragraph that begins at page 34, line 30 
with "The OEVI assistant scenario imposes largely 

"The FIG. 15a illustration shows, for example, how rhythmic flow 1511 
may be applied to user input, e.g., a determinable manner in which commands may be 
input while working with lists 1501. Factors for determining rhythmic flow (e.g., and 
correspondingly determining command portions in the illustrate4d example include degree 
of finality, break in momentum or new display." 

The following new paragraph is ADDED after the paragraph that begins at page 22, line 14 
with "Consider, for example, the OEVI-like assistant scenario . . .": 

"FIG. 15b, for example, illustrates how switching or re-directing may be 
applied to machines such as multiple underlying interface portions (e.g., multiple windows 
1502a-b) or different systems (e.g., remote and local systems 1521a-b. 

Add the following new paragraph after the paragraph that begins at page 67, line 2 with 
"FIG. 7e illustrates how an exemplary enhancement engine . ..": 

"(FIG. 15c, for example, illustrates how an enhanced command may be 
used for switching between multiple interface portions, for example, lists of different types 
(e.g., multiple lists 1503a, 1503b that fiirther may include multiple highlighting) or in 
conjunction with underlying interface portions, for example, divided forms, e.g., 1530, 
which includes data entry and list portions 1531, 1532 or not highlighted lists.)" 
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The following new paragraph is ADDED after the paragraph that begins at page 70, line 15 
with "Analyzer-filters 823-825 can, for example, be used...": 

"FIG. 15d, for example, illustrates an interfacing system 1504 embodiment 
in which analyzing, controlling or filtering received information is conducted using control 
supressor 725 of application support engine 614, analyzer/ filters 822-926 of security 
engine 622, control suppressor 744 of enhancement engine 616 and substitutes/ parameters. 
In this example, machine-m 1504b may receive portions of the received information or 
substitute portions, or machine-m 1504b may receive only not-switched output while 
machine-n 1504c may received switched (e.g., private) information." 

The following new paragraph is ADDED after the paragraph that begins at page 27, line 4 
with "Tying command recitation or handling. . 

"FIG. 15g illustrates an example of received user input processing an 
interfacing system 1507. In this example, interpreter 1571 provides for trapping user input 
portions, which in this example include "mis-recitations", such that the interpreter provides 
no output or processed output. Interpreter 1571 may also provide for other handing of the 
received mis-recitation, as indicated generally by the dotting of element 1572." 

The following 2 new consecutive paragraphs are ADDED after the paragraph that begins at 
page 73, line 27 with "It will be appreciated that the utilization...": 

"The FIG. 16 flowchart illustrates a method of conducting voice 
information. As shown, block 1602 includes receiving information, block 1604 includes 
receiving applicable task data and designations and block 1 606 includes determining any 
pre, concurrent and/or post command extensions. Block 1608 fiirther includes determining 
any task, data, designation or related feedback. Block 1610 includes effectuating machine 
operation corresponding to the task, data, designations and extensions. Block 1612 
includes effectuating any additional feedback. 

The FIG. 17 flowchart illustrates a method for conducting user 
identification and interaction interpretation. As shown, block 1702 includes determining at 
least one of a comparative and a specific user identification of a user and block 1 704 
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includes associating the user identification with a user (input) device. Block 1 706 further 
includes determining applicable ones of user, group, security and interaction information 
corresponding to the user identification. Block 1 708 includes receiving user input and 
block 1710 includes interpreting the received user input according to the association and 
corresponding one or more of user, group, security and interaction information. Block 
1712 further includes intermittently determining at least comparative user identification 
e.g., potential substitution or other time/event occurrence." 

The 2 paragraphs that begin at Specification page 49, line 1 are AMENDED as follow^s: 

"Operation KB 407 includes existing command KB 407a, underlying machine KB 
407b speech engine KB 407c and user/group KB 407d. Existing command KB 407a and 
underlying machine KB 407b include elements for enabling communication and operability 
command execution with known or new machines. Existing command KB 407b includes more 
generally applicable and communication information, such as pervasive commands 471, e.g. OS 
commands, general functions (see examples below), common commands (such as PC save, load 
and edit commands), and so on. Also included are connection and OS information 472, 
command/machine distribution information 473 (e.g. associated command/data structures, 
memory/storage locations, and so on), communication and command delivery or other 
communication protocols 474, and other information 474475. Underlying machine KB 407b 
includes machine capabilities 481 476, available controls 482 477, available feedback 483, 
available parameters 484 47^, memory map information 485 479 (e.g. data/attribute or control 
layout of programs/devices), enhancement information 486 480 (e.g. desirable enhancements and 
implementation), and linking information 487 4^ (e.g. for below discussed overlaying or other 
multi-machine capabilities). Other machine interfacing information 488 482 can also be utilized. 

Speech engine KB 407c includes information pertaining to communicating with a 
particular speech recognition or synthesis engine, such as available control mechanisms 491 
(e.g. information handling, synthesis speed/pitch, and so on), command parameters 492 (e.g. for 
other available controls or control variations), form information 493 and other information 494. 
(Note that attributes of other event/gesture processing engine peculiarities can also be utilized, 
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or one or more front or back end knowledge bases can also be utilized and other information 
can be utilized in accordance with a particular application.)" 

The paragraph beginning at Specification page 58, line 5 is AMENDED as follows: 

"Enhancements 503 can provide peripheral capabilities that can further be related to other 
command portions. For example, an enhancement of "Send an email to C" might identify B as 
a current user, activate B's PC, PIM, etc. (e.g. if remotely initiated), run an email-handling 
program or applet (if various programs might be run), initiate a new email message, address the 
message, and prepare (e.g. via curser repositioning) for an email message subject. Thus, a next 
user command can simply enter the subject (with corresponding enhancements of advancing the 
curser to the email message body, providing for additional addressing, etc) or redirect operation 
to a less-often utilized successive capability. See, for example, FIGS. 3a-c, 10b, 12a, 13b-c and 
14." 

The paragraph beginning at page 65, line 22 is amended as follows: 

"FIG. 7d illustrates how an exemplary designation engine 615 provides for receiving 
designations from (typically) command engine 612 (FIG. 7b) and determining therefrom or 
"resolving" available references (typically in accordance with application support engine 614 of 
FIG. 7b or one or more of engines 621a-627 of FIGS. 8a-8g. As shown, various "designation 
modes" can be provided as pertaining to a particular user, application, machine, etc. or be 
provided on a more global basis, as with the OEVI. Designation control 731 provides an 
overall designation control mechanism, as already discussed. Explicit designation resolver 732, 
implied designation resolver733, operation resolver 734, data resolver 735 and application- 
machine resolver 736 more specifically provide for resolution of explicit, implied, tool, data and 
application/machine specific designations respectively. Command input resolver 737 
provides for resolving available or applicable input commands, dictation or other data types 
(e.g. according to root, base, designation, extension, enhancement, user, group, user/group 
ID/status, history, context, and so on, for example, as was already discussed. 

The paragraph beginning at page 70, line 4 have been amended as follows: 
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"Analyzer- filters 823-826 and analyzer 822 822 826 facilitate security more particularly 
related to the freedom provided by conversance or otherwise more "free-form" I/O. That is, 
rather than treating user/ machine input treated as generic, analyzer-filters 823-826 and analyzer 
822 822 826 provide for determining received information content or content types and, upon 
determining such information (e.g. a particular designation, specificity, confidential data 
element, etc.), provide for correspondingly removing, modifying or replacing the input 
element(s). Such analysis/determining can, for example, be conducted via parsing input, stored 
or for-output information respectively, and comparing the information to a stored or otherwise 
ascertainable element set; such element or elements can be replaced, deleted or modified in 
accordance with corresponding elements or other processing (e.g. users, purposes, contexts, 
tasks, applications, machines, providing feedback, etc.) (Other suitable mechanisms can also be 
used.). 
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